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Traditional  interoperability  testing  focuses  on  the  operational  effectiveness  of  preplanned 
information  exchanges  to  and  from  the  capability  being  tested  as  well  as  functionality  of  the 
capability  to  perform  its  mission  objectives.  As  the  Department  of  Defense  continues  to  migrate  to  a 
net-centric  architecture,  standalone  systems  will  be  replaced  with  service-based  capabilities  deployed 
in  various  enterprises.  In  this  context ,  a  capability  inherits  both  the  risks  and  requirements  associated 
with  that  enterprise.  The  relationship  between  a  capability  and  the  enterprise  on  which  it  is  deployed 
is  symbiotic  and,  as  such,  requires  an  evaluation  of  capability  functionality  as  well  as  the  ability  of 
the  enterprise  to  support  the  capability’s  mission-driven  business  processes. 
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The  Department  of  Defense  (DoD)  test 
community  has  a  long  history  of  “pro- 
gram-level”  interoperability  testing. 
Current  methods  focus  on  assessing 
information  exchanges  for  operational 
effectiveness,  but  do  not  include  an  assessment  of  the 
enterprise  architecture  components.  The  enterprise  is  a 
community  of  systems  and  services  (e.g.,  people, 
organizations,  and  technology)  that  are  interdependent 
and  must  coordinate  functions  and  share  information  in 
support  of  a  common  mission  or  a  set  of  related  missions. 
This  test  philosophy  cannot  support  a  net-centric  DoD 
where  decision  making  is  based  on  tiered  accountability 
and  federated  governance.  In  the  net-centric  enterprise, 
the  line  between  “mine”  and  “yours”  no  longer  exists. 
Investments  are  still  made  based  on  capability  gaps,  but 
only  after  the  benefits  of  reuse  and  loose  coupling  are 
fully  exploited  through  the  application  of  service- 
oriented  architecture  (SOA)  best  practices.  The  tester 
must  look  at  the  enterprise  holistically  and  determine 
when  it  comes  to  net  centricity,  how  are  we  doing? 

Introduction 

The  Defense  Information  Enterprise  Architecture 
(DIEA)  vl.O  (April  2008)  describes  the  DoD  net- 
centric  vision  as  follows1: 


“ To  function  as  one  unified  DoD  Enterprise, 
creating  an  information  advantage  for  our  people 
and  mission  partners  by  providing: 

•  A  rich  information  sharing  environment 
in  which  data  and  services  are  visible, 
accessible,  understandable,  and  trusted  across 
the  enterprise; 

•  An  available  and  protected  network  infra¬ 
structure  (the  Global  Information  Grid 
(GIG))  that  enables  responsive  informa¬ 
tion-centric  operations  using  dynamic  and 
interoperable  communications  and  comput¬ 
ing  capabilities.  ” 

As  testers,  how  do  we  ensure  that  the  DoD  does,  in 
fact,  operate  as  one  seamless  and  interoperable  enter¬ 
prise,  even  while  implementing  a  tiered  accountability 
decision  model  and  a  federated  governance  structure? 

JITC  proposes  a  test  approach  ( Figure  1)  that 
includes  all  of  the  components  of  an  enterprise  that 
are  required  to  ensure  success.  However,  not  all 
requirements  in  the  test  approach  will  apply  to  every 
enterprise.  The  test  approach  should  therefore  be 
tailored  to  meet  the  needs  of  each  unique  enterprise. 
This  approach  aligns  with  DIEA  objectives,  imple- 
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Connectivity 


Local  Services  Core  Services 

Figure  1.  Net-centric  enterprise  testing  approach. 


ments  best-of-breed  practices  from  industry  and 
academia,  and  provides  value-added  information  to 
DoD  capability  portfolio  managers,  enterprise  archi¬ 
tects,  and  program  managers  for  decision  making.  This 
approach  provides  technical  rigor,  flexibility,  and 
scalability  required  to  ensure  testing  provides  value- 
added  information  within  the  cost  and  schedule 
constraints  of  rapid  acquisition  initiatives. 

Connectivity 

Connectivity  refers  to  the  hardware  and  software 
implementations  that  enable  connection  between 
clients  and  servers,  between  services,  and  among 
networks  (Figure  1).  These  implementations  comprise 
the  enterprise  network  infrastructure  and  support  the 
mission-critical  and  mission-enabling  (nonmission- 
critical)  information  exchanges  between  capabilities 
on  the  enterprise  to  include  clients,  servers,  services, 
and  networks. 


Requirements 

Compliance  with  the  DIEA  requires  that  the 
connectivity  of  the  enterprise  support  capability 
missions  by  providing  secure,  dynamic,  computing- 
platform  agnostic  and  location-independent  data 
storage,  real-time  provisioning,  allocation  of  shared 
resources,  and  access  to  shared  spaces  and  information 
assets  within  the  mission-required  Quality  of  Service 
(QoS)  parameters. 

Metrics 

The  following  measurements  comprise  the  suggest¬ 
ed  minimum  set  of  metrics  needed  to  support  an 


enterprise  connectivity  assessment.  However,  this  list 
should  be  tailored  to  accommodate  each  unique 
enterprise  by  either  deleting  nonapplicable  metrics  or 
adding  new  metrics. 

Operational  availability  (A0).  Testers  will  review  logs 
captured  with  an  enterprise  service  management  tool 
and  verify  the  operational  availability  of  the  network 
connections.  This  is  determined  by  using  the  opera¬ 
tional  availability  equation  \A0  =  Mean  Time  between 
Failure  (MTBF)/MTBF  +  Mean  Time  to  Repair 
(MTTR)]  and  is  represented  as  a  percentage.2 

Lowest  throughput  (in  GB/s).  This  metric  indicates 
availability  of  the  network  by  measuring  the  lowest 
throughput  speed  at  all  nodes  during  normal  message 
request  load  on  the  enterprise.  A  network  experienc¬ 
ing  lowered  throughput  may  result  in  slow  message 
exchanges  that  delay  service  access  or  interruptions  in 
service  access,  which  prevent  function.  Using  a 
network  loading  tool,  testers  will  simulate  network 
load  with  a  normal  random  distribution  over  time  and 
measure  the  throughput  speeds  at  all  nodes.  If  the 
lowest  throughput  speed  does  not  meet  mission- 
critical  threshold  requirements,  then  the  enterprise 
cannot  support  the  mission. 

Bandwidth  usage.  This  metric  indicates  overall 
network  load  capacity.  Bandwidth  usage  data  are 
collected  by  enterprise  service  management  tools  and 
enterprise  logs.  The  data  traffic  is  then  compared  with 
the  total  data  resources  available.  Testers  will  verify 
that  the  bandwidth  used  by  the  capabilities  on  the 
enterprise  do  not  exceed  the  set  limits  defined  in  the 
foundational  documentation.  Measurements  should 
focus  on  high  traffic  services  and  large  data  output 
services  using  realistic  network  load  distributions  over 
time. 

Core  Services 

Core  services  are  ubiquitous,  common  solution 
services  that  provide  capabilities  essential  to  the 
operation  of  the  enterprise.3  They  are  infrastructure- 
type  capabilities  that  support  multiple  key  consumers. 
Examples  of  core  services  include: 

•  security  and  authentication  services, 

•  orchestration  services, 

•  load  monitoring  services, 

•  load  balancing  services, 

•  messaging  services, 

•  service  configuration  monitoring  tools, 

•  enterprise  service  management  tools, 

•  enterprise  test  tools, 

•  enterprise  service  bus  capabilities. 
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Core  service  testing  will  target  the  core  services  that 
are  available  on  the  enterprise  and  will  focus  on  their 
net-centric  performance  and  design  in  support  of 
capability  mission  requirements. 

Requirements 

Compliance  with  the  DIEA  requires  that  the  core 
services  deployed  on  a  given  enterprise  support  the 
mission  by  implementing  a  loosely  coupled  architecture 
that  is  visible,  accessible,  understandable,  and  trusted 
by  both  anticipated  and  unanticipated  users.  Core 
services  must  support  mission  requirements  even 
during  Disconnected,  Intermittent,  or  Limited  (DIL) 
bandwidth  conditions  and  ought  to  provide  Network 
Operations  (NetOps)  related  data,  such  as  performance 
and  availability,  to  ensure  compliance  with  GIG  service 
level  agreements  (SLAs). 

Metrics 

The  following  measurements  comprise  the  suggest¬ 
ed  minimum  set  of  metrics  needed  to  support  an 
enterprise  core  services  assessment.  However,  this  list 
should  be  tailored  to  accommodate  each  unique 
enterprise  by  either  deleting  nonapplicable  metrics  or 
adding  new  ones. 

Operational  availability  (A0).  Testers  will  review  logs 
captured  with  an  enterprise  service  management  tool 
and  verify  the  operational  availability  of  enterprise  core 
services.  This  is  determined  by  using  the  operational 
availability  equation  ( A0  =  MTBF/MTBF  +  MTTR) 
and  is  represented  as  a  percentage.4 

Maximum  latency  (response  time)  for  average 
request  (in  ms).  Time  difference  between  requestor’s 
service  request  and  service  response  (measured  from 
time  of  service  request  [received  at  service  provider]  to 
time  of  response  [sent  from  service  provider]).  Testers 
will  test  core  service  maximum  latency  to  determine 
whether  mission-critical  threshold  requirements  for 
response  times  have  been  met. 

Idempotency  (in  a  stateless  client).  This  metric 
indicates  the  uniformity  of  the  responses  from  the 
core  service.  If  the  service  client  is  stateless,  then  the 
response  received  after  executing  a  service  call  should 
be  the  same  no  matter  how  many  times  the  service  call 
is  executed.5  Testers  should  send  a  statistically 
significant  number  of  identical  service  calls  (messages) 
to  the  core  service  and  verify  that  identical  responses 
are  received. 

Data  accuracy.  Core  services  that  provide  data  should 
provide  a  quantifiable  measurement  of  data  accuracy.  A 
capability  could  have  varying  degrees  of  accuracy 
requirements.  Enterprise  core  services  must,  as  a 


threshold  requirement,  support  the  highest  level  of 
mission-critical  accuracy  required  by  enterprise  capa¬ 
bilities  that  will  utilize  those  core  services. 

Maximum  size  of  user  domain.  This  metric  in¬ 
dicates  scalability  of  service,  i.e.,  how  well  the  core 
services  can  support  the  user  domains  required  by 
the  sum  total  of  all  missions  executed  in  the  enter¬ 
prise. 

Maximum  number  of  simultaneous  users.  This 
metric  identifies  the  maximum  number  of  concurrent 
users  performing  “normal”  operations  beyond  which  A0 
or  throughput  falls  below  acceptable  levels.  It  indicates 
scalability  of  core  service  and  consistency  of  perfor¬ 
mance  under  varying  load  conditions.  The  threshold 
requirement  for  each  enterprise  core  service  must 
represent  the  sum  of  the  average  number  of  concurrent 
users  required  by  all  supported  capabilities  deployed  on 
the  enterprise. 

Local  services 

Local  services  are  application-type  capabilities  that 
provide  a  function  in  support  of  an  operational 
requirement  or  mission.  Local  services  may  vary  from 
extremely  small  bits  of  capability  (provides  a  map)  to 
large  capabilities  drawn  from  service  enabling  a  stove- 
piped  legacy  system. 

Local  service  testing  focuses  on  both  the  function¬ 
ality  of  capabilities  on  the  enterprise,  as  well  as 
compliance  with  inherited  enterprise  requirements. 

Requirements 

Compliance  with  the  DIEA  requires  that  the 
local  services  deployed  on  a  given  enterprise  sup¬ 
port  the  mission  by  implementing  a  loosely  coupled 
architecture  that  is  visible,  accessible,  understandable, 
and  trusted  by  both  anticipated  and  unanticipated 
users.  Local  services  must  provide  access  to  authorita¬ 
tive  data  assets,  services,  and  applications,  and  be 
accessible  to  all  authorized  users  except  where  limited 
by  law,  policy,  security,  classification,  or  operational 
necessity.  Local  services  must  support  graceful  degra¬ 
dation  of  capability  and  performance  during  DIL 
conditions  and  ought  to  provide  NetOps  related  data, 
such  as  performance  and  availability,  to  ensure 
compliance  with  GIG  SLAs. 

Metrics 

The  following  measurements  comprise  the  suggest¬ 
ed  minimum  set  of  metrics  needed  to  support  an 
enterprise  local  services  assessment.  However,  this  list 
should  be  tailored  to  accommodate  each  unique 
enterprise  by  either  deleting  nonapplicable  metrics  or 
adding  new  metrics. 
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Service  visibility.  Testers  will  ensure  that  local  service 
is  registered  in  the  enterprise  service  registry  and  is 
“discoverable”  with  an  intuitive  keyword  search  using 
the  enterprise’s  federated  search  capability. 

Service  accessibility.  Testers  will  ensure  that  local 
service  has  written  policy  listing  actions  necessary  to 
gain  transparent  machine-to-machine  access  to  ser¬ 
vices  via  user  level  credentials,  system  level  creden¬ 
tials,  or  trust  relationships  (e.g.,  SLAs).  This  includes 
users  who  are  anticipated  (i.e.,  known  users  with 
specific  missions  that  have  been  granted  access  to  the 
system),  unanticipated  (i.e.,  users  without  specifically 
defined  missions  who  have  been  granted  access  to  the 
system),  and  unauthorized  users  (i.e.,  users  without 
access  to  the  system).  Policy  must  be  registered  in 
the  service’s  submission  package  located  in  the  DoD 
Metadata  Registry,  and  policy  must  be  enforced  as 
written. 

Service  understandabiiity.  Testers  will  ensure  that  the 
local  service  Web  Services  Description  Language 
correctly  executes  service  operations  and  any  commu¬ 
nity  of  interest  (COI)  or  Enterprise  mandated 
vocabularies  and  schemas  are  used  and  implemented 
correctly. 

Service  reuse.  Existing  enterprise  services  and  end- 
user  interfaces  shall  be  used  whenever  possible, 
practical,  and  appropriate  instead  of  recreating  those 
assets.6 

Functional  requirements.  Local  services  used  by  the 
warfighter  or  capability  must  provide  functional 
capabilities  as  described  in  the  U.S.  Joint  Forces 
Command-maintained  Joint  Common  System  Func¬ 
tions  List  QCSFL).  The  JCSFL  provides  a  common 
lexicon  for  system  Command  and  Control  (C2) 
functionality,  including  the  traceability  of  Military 
Service  C2  functions  to  their  joint  equivalent,  for 
interoperability  and  comparative  analyses.  The  JCSFL 
describes  the  C2  functionality  of  any  platform, 
program  of  record,  system,  subsystem,  component,  or 
application  that  provides  such  functionality.  The 
JCSFL  also  contains  intelligence,  surveillance,  and 
reconnaisance  functions  and  will  be  updated  to  include 
net-centric  communications  functionality,  as  deter¬ 
mined  by  the  net-centric  Capability  Portfolio  Manag¬ 
er.  Support  functions,  including  those  for  maintenance, 
logistics,  medical,  personnel,  training,  etc.,  will  be 
included  in  future  revisions.7 

Operational  availability  (A0).  Testers  will  review 
logs  captured  with  an  enterprise  service  management 
tool  and  verify  the  operational  availability  of  local 
services  to  each  client  capability.  This  is  determined  by 


using  the  operational  availability  equation  (. Aa  = 
MTBF/MTBF+MTTR)  and  is  represented  as  a 
percentage.8 

Maximum  latency  (response  time)  for  average 
request  (in  ms).  This  metric  is  the  time  difference 
between  requestor’s  service  request  and  service  response 
(measured  from  time  of  service  request  [received  at 
service  provider]  to  time  of  response  [sent  from  service 
provider]).  Testers  will  test  local  service  maximum 
latency  to  determine  whether  mission-critical  threshold 
requirements  for  response  times  have  been  met. 

Data  accuracy.  Local  services  that  provide  data  should 
provide  a  quantifiable  measurement  of  data  accuracy. 
Local  services  must,  as  a  threshold  requirement, 
support  the  highest  level  of  mission-critical  accuracy 
required  by  the  client  capability. 

Maximum  size  of  user  domain.  This  metric  indicates 
scalability  of  service,  i.e.,  how  well  the  local  service  can 
support  the  user  domains  required  by  the  sum  total  of 
all  missions  executed  in  the  enterprise. 

Maximum  number  of  simultaneous  users.  Identifies 
the  maximum  number  of  concurrent  users  performing 
“normal”  operations  beyond  which  A0  or  throughput 
falls  below  acceptable  levels.  This  metric  indicates 
scalability  of  local  service  and  consistency  of  perfor¬ 
mance  under  varying  load  conditions.  The  threshold 
requirement  for  the  local  service  must  represent  the 
sum  of  the  average  number  of  concurrent  users 
required  by  all  supported  capabilities  deployed  on  the 
enterprise. 

Graceful  capability  degradation.  This  metric  identi¬ 
fies  local  service  ability  to  implement  graceful  degra¬ 
dation  capabilities  as  outlined  by  mission  requirements. 
Mission  requirements  ought  to  specify  capabilities 
required  under  varying  levels  of  DIL  bandwidth 
conditions.  Testers  should  evaluate  local  services  under 
conditions  specified  to  ensure  that  threshold  capability 
requirements  are  met. 

Data 

Data  testing  will  target  the  data  assets  that  are 
shared  within  the  enterprise  and  will  focus  on  the 
ability  of  those  assets  to  support  mission-critical 
threshold  capability  requirements.  There  are  two  types 
of  data:  content  data  and  metadata. 

Content  data  are  data  provided  by  a  capability  that 
provides  information  usable  by  other  capabilities  or 
users.  Content  data  address  the  needs  of  the  COIs  and 
users  or  warfighters  directly.  A  capability  generally 
generates,  transforms,  stores,  or  consumes  content 
data. 
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Metadata  are  data  that  describe  the  characteristics  of 
the  capability  or  data  exposed  on  the  enterprise. 
Metadata  generally  describe  content  data  and/or 
services  that  are  available  for  consumption,  e.g.,  what 
standards  the  service  or  data  asset  follows,  how  to  use 
the  service  or  data  asset  (for  machine-to-machine 
interface),  and  how  to  discover  the  service  or  data. 

Requirements 

Compliance  with  the  DIEA  requires  that  the  data 
deployed  on  a  given  enterprise  support  the  mission  by 
being  visible,  accessible,  understandable,  and  trusted  by 
both  anticipated  and  unanticipated  users.  Data  should 
follow  the  syntax  and  semantics  as  defined  by  the 
associated  community  of  interest  and  should  be 
appropriately  tagged  using  the  enterprise  standard  for 
discovery  metadata  (DoD  Discovery  Metadata  Speci¬ 
fication). 

Metrics 

The  following  measurements  comprise  the  suggested 
minimum  set  of  metrics  needed  to  support  an  enterprise 
data  assessment.  However,  this  list  should  be  tailored  to 
accommodate  each  unique  enterprise  by  either  deleting 
nonapplicable  metrics  or  adding  new  metrics. 

Data  visibility.  Testers  will  ensure  that  discovery 
metadata  are  registered  in  an  Enterprise  Catalog  in 
accordance  with  DDMS,  thus  making  it  discoverable 
within  the  targeted  enclave.  Testers  will  ensure  that  data 
are  “discoverable”  with  an  intuitive  keyword  search 
using  the  enterprise’s  federated  search  capability. 

Data  accessibility.  Testers  will  ensure  that  Feder¬ 
ated  Search  results  provide  active  link  (e.g.,  Uni¬ 
form  Resource  Locator)  that  points  to  the  specified 
data  asset  within  the  targeted  security  enclave.  Testers 
will  ensure  that  the  data  provider  has  written  policy 
listing  actions  necessary  to  gain  transparent  user 
access  to  the  data  via  user  level  credentials,  system 
level  credentials,  or  trust  relationships  (e.g.,  Access 
Control  List).  This  includes  users  who  are  anticipated 
(i.e.,  known  users  with  specific  missions  that  have 
been  granted  access  to  the  system),  unanticipated 
(i.e.,  users  without  specifically  defined  missions  who 
have  been  granted  access  to  the  system),  and 
unauthorized  users  (i.e.,  users  without  access  to  the 
system).  Policy  information  must  be  registered  in  an 
enterprise  catalog ,  include  the  steps  by  which  a  user  may 
request  access  to  the  data,  and  be  available  within  “2 
clicks”  from  the  active  link  provided  by  Federated 
Search. 

Data  understandability.  Testers  will  ensure  that  data 
are  navigable  within  the  limitations  of  the  interface,  are 


labeled  with  meaningful  labels,  are  conveyed  effective¬ 
ly,  and  use  commonly  understood  language  that 
conforms  to  COI-approved  vocabularies. 

Semantic  reuse.  Semantic  vocabularies  shall  reuse 
elements  of  the  DoD  Intelligence  Community  (IQ- 
Universal  Core  information  exchange  schema. 

Data  reuse.  Existing  enterprise  data  shall  be  used 
whenever  possible,  practical,  and  appropriate,  instead 
of  recreating  those  assets.10 

Data  accuracy.  Data  should  provide  a  quantifiable 
measurement  of  accuracy.  Data  that  are  provided  as  a 
service  should  maintain  source  level  of  accuracy  in 
accordance  with  mission-critical  threshold  require¬ 
ments. 

Data  refresh  rate.  Data  must  maintain  a  refresh  rate 
that  is  compliant  with  the  threshold  mission  require¬ 
ments  for  the  client  capability. 

Graceful  degradation.  Data  must  be  accessible  during 
DIL  bandwidth  conditions.  Data  redundancy  should 
be  made  available  through  the  use  of  local  caching  and 
data  storage.  Data  should  be  appropriately  tagged  with 
“age”  or  “time  of  last  refresh”  information  so  that  the 
user  or  warfighter  is  aware  of  the  currency  of  the  data 
for  decision  making  purposes. 

Processes  and  workflows 

A  process  is  a  composition  of  one  or  more  types  of 
services  that  are  capable  of  accomplishing  a  particular  part 
of  a  mission  objective  (Figure  1).  For  example,  “perform 
capability  A,  translate  the  results,  then  perform  capability 
B”  is  a  process.  A  workflow  is  a  specific  composition  of 
processes  and  services  that  will  accomplish  a  mission 
objective.  For  example,  “service  A  calls  translation  service 
AB,  which  calls  service  B”  is  a  workflow. 

Processes  and  workflows  testing  will  target  the 
business  processes  that  combine  to  form  workflows  to 
accomplish  capability  mission  objectives. 

Requirements 

Process  and  workflow  requirements  should  be 
derived  from  joint  mission  threads  developed  by  the 
user  representative  for  a  given  capability.  Mission 
threads  should  provide  operational  activities,  tasks,  and 
required  performance  characteristics  needed  to  meet 
threshold  mission-critical  requirements.  Mission 
threads  will  be  decomposed  into  the  materiel  and 
nonmateriel  solutions  required  to  execute  the  thread. 
The  processes  and  workflows  required  to  execute  a 
given  mission  are  best  represented  using  dynamic 
modeling  techniques  such  as  business  process  modeling 
notation. 
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Metrics 

The  following  measurements  comprise  the  suggest¬ 
ed  minimum  set  of  metrics  needed  to  support  an 
enterprise  processes  and  workflows  assessment.  How¬ 
ever,  this  list  should  be  tailored  to  accommodate  each 
unique  enterprise  by  either  deleting  nonapplicable 
metrics  or  adding  new  metrics. 

Operational  effectiveness.  Operational  effectiveness  is 
the  overall  degree  of  mission  accomplishment  of  a  system 
when  used  by  representative  personnel  in  the  environ¬ 
ment  planned  or  expected  for  operational  employment  of 
the  system  considering  organization,  doctrine,  tactics, 
survivability,  vulnerability,  and  threat.  The  evaluation  of 
operational  effectiveness  is  linked  to  mission  accom¬ 
plishment.  The  early  planning  for  the  evaluation  should 
consider  any  special  test  requirements,  such  as  the  need 
for  large  test  areas  or  ranges  or  supporting  forces, 
requirements  for  threat  systems  or  simulators,  new 
instrumentation,  or  other  unique  support  requirements. 

Operational  suitability.  Operational  suitability  is  the 
degree  to  which  a  system  can  be  satisfactorily  placed  in 
field  use,  with  consideration  given  to  reliability, 
availability,  compatibility,  transportability,  interopera¬ 
bility,  wartime  usage  rates,  maintainability,  safety, 
human  factors,  manpower  supportability,  logistics  sup- 
portability,  documentation,  training  requirements,  and 
natural  environmental  effects  and  impacts.  Early  plan¬ 
ning  for  the  suitability  evaluation  should  include  any 
special  needs  for  number  of  operating  hours,  environ¬ 
mental  testing,  maintenance  demonstrations,  testing 
profiles,  usability  of  developmental  testing  data,  or  other 
unique  test  requirements.  Operational  suitability  should 
be  evaluated  in  a  mission  context  to  provide  meaningful 
results.  For  example,  maintaining  a  required  Operations 
Tempo  over  an  extended  period  while  conducting 
realistic  missions  gives  insight  into  the  interactions  of 
various  suitability  factors,  such  as  the  ability  to  maintain 
stealth  features  during  sustained  operations. 

Operational  survivability.  Operational  survivability  is 
the  degree  to  which  a  capability  is  able  to  resist  or 
recover  from  detrimental  effects.  Measurement  time 
frames  should  be  from  the  start  of  unavailability  to  the 
time  when  service  is  restored.  The  enterprise  should 
have  automated  tools  in  place  to  restore  service 
automatically  after  service  is  lost. 

Summary 

This  enterprise-level  test  approach  provides  the  basis 
for  evaluating  a  net-centric  enterprise  by  examining  the 
individual  capabilities  in  context  with  the  components 
of  their  parent  enterprise:  connectivity,  core  services, 
local  services,  data,  and  processes  and  workflows. 


“Program-level”  interoperability  testing  does  not 
reveal  problems  that  will  occur  as  a  result  of  the 
growing  intricacy  of  client-service  dependencies, 
changing  interface  requirements,  and  resource  scaling 
issues  as  the  enterprise  (and  its  resident  service,  data 
and  infrastructure  assets)  matures  and  multiplies. 
“Program-level”  interoperability  testing  also  does  not 
reveal  the  benefits  of  reuse  and  loose  coupling  that  may 
be  achieved  by  the  enterprise  through  the  application 
of  SOA  best  practices. 

In  contrast,  this  enterprise-level  test  approach  will 
expose  interservice  dependencies  and  shortcomings  and 
highlight  the  benefits  of  existing  enterprise  infrastruc¬ 
ture  and  SOA  governance  assets  that  promote 
efficiency,  enable  development,  and  manage  growth 
of  net-centric  technologies.  □ 
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